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A method for selecting a suitable subspace for discriminating signal components through an 
oblique projection is proposed. The selection criterion is based on the consistency principle intro- 
0^ ■ duced by M. Unser and A. Aldroubi and extended by Y. Elder. An effective implementation of this 

principle for the purpose of subspace selection is achieved by updating of the dual vectors yielding 
the corresponding oblique projector. 
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Oblique projectors are of assistance to signal processing applications [1—7], in particular due to their 

<N 

ability to discriminate signal components lying in different subspaces. Thereby, as discussed in [1], 

. oblique projectors are suitable for filtering structured noise. Let us suppose for instance that a given 
O . 

signal /, represented mathematically as an element of a vector space H, is produced by the super- 

o , 

position of two phenomena, i.e. / = fi + f% where fi belongs to a subspace S\ C TL and fa belongs 
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to subspace 52 C 7i. Provided that Si n 52 = {0} we can obtain from / the component f\ by an 
' oblique projection onto S\ along 52, which maps fi to zero without altering f\. The procedure is 

X 

straightforward and effective if the corresponding subspaces S\ and 52, such that S\ fl 52 = {0}, are 
known [1]. Nevertheless, this may not be always the case. In this letter we address the problem of 
selecting the appropriate subspace Si, from the spanning set of a larger subspace, in order to fulfil the 
condition Si H 52 = {0} assuming that 52 is known and fixed. 

Given a signal, our strategy for the selection of the representation subspace is in the line of Matching 
Pursuit (MP) methodologies [8-12] and is made out of two ingredients i) the sampling/reconstruction 
consistency requirement introduced in [2] and extended in [6] ii) a recursive procedure for adapting 
the dual vectors giving rise to the corresponding oblique projector [13]. It will be shown here that 
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the latter yields an effective implementation of a selection criterion that we base on the consistency 
principle. 

The letter is organized as follows: Sec [2] introduces the general framework and discusses the 
ingredients of the approach. Namely, the consistency principle and the recursive updating of the 
measurement vectors for achieving the required oblique projection. The Oblique Matching Pursuit 
strategy is introduced in Sec Its implementation is discussed in Sec H] along with a numerical 
example. The conclusions are drawn in SecO 

2 The consistency principle and stepwise updating of measurement 
vectors 

We represent a signal / as an element of an inner product space that is assumed to be finite dimensional. 
The square norm is computed as ||/|| 2 = (/,/}, where the brackets denote the corresponding inner 
product and we define the inner product in such a way that if c is a complex number (cf,g) = c*(f,g), 
with c* the complex conjugate of c. Measurements of a signal / (also called samples) will be represented 
as linear functionals. Thus a set of, say k, sampling vectors i = 1, . . . , k provides us with a set 
of k measurements on / given by the inner products (wf, /), i = l,...,k. The superscript k is used 
to indicate that to reconstruct the signal we will need to modify the measurement vectors w k if an 
additional measure is considered. From the sampling measurements we can construct an approximation 
f k of / using a set of reconstruction vectors Vi, i = l,...,k. The consistency principle introduced 
in [2] states that the reconstruction f k from (wf, /), i = 1, . . . , k should be self-consistent in the sense 
that if the approximation is sampled with the same vectors the same samples should be obtained. 
In other words, a consistent reconstruction must satisfy: (w k ,f k ) = (w k ,f), i = 1, ...,k. This 
requirement has been considered further in [6] where it is proved that: if the reconstruction vectors 
V{, i = 1, . . . , k span a subspace Vk and the sampling vectors w k , i = 1, . . . , k span a subspace Wk such 
that its orthogonal complement W 1 - satisfies VtHW 1 = {0}, then f k is a consistent reconstruction of 
f if and only if f k is the oblique projection of f onto Vk along . We represent the corresponding 
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oblique projector as E Vk y V ±. Hence, it is endowed with the following properties i) w± = E VkW ±, 
ii) E VkW ±v = v, for any v <G V k hi) E VkW ±w = 0, for any w € W -1 . Given the conditions of the 
above statement, the unique consistent approximation of / is therefore f k = E VkW ±f. The oblique 
projector can be expressed as E VkW ± = Yli=i v i( w h ') where (wf,-) indicates that Ey W ± acts by 
performing inner products as in E VkVV ±f = Yli=i v i( w ii /}■ Explicit equations for updating an oblique 
projector when a new pair of reconstruction/measurement vectors is to be considered are given in [13]. 
As will be discussed in the next sections, for the purpose of this contribution we can restrict the 
measurement vectors to be lineally independent. Hence the vectors w k+1 yielding oblique projectors 
along W ± onto nested subspaces V k +i = V k + v k+ \ = spanl^}^ 1 can be inductively obtained as 
follows: 

Construct vectors u,- L = V{ — Py V ±Vi, with Py V ± the orthogonal projector onto W ± . From w\ = yf^jp 
every time a new vector is needed compute it, and update the previous ones, through the equations [13]: 



f +1 = w^-w k k Xl(u k+1 M), i = l,...,fc (1) 



= IiShTF' Qk+i = u k+1 - P Wh u k+1 , (2) 

where P\y k is the orthogonal projector onto W k = span{uj}^ =1 . It should be noticed that Vfc+i + W" 1 = 
Wfc+i © W -1 -, with © indicating the orthogonal sum and + the direct sum. 

In the next section we introduce a method for stepwise selection of the measurement vectors aiming 
at finding a subspace V k for reconstruction such that V k H W ± = {0}. 

3 Oblique Matching Pursuit (OBLMP) 

Matching Pursuit strategies for signal representation evolve by stepwise selection of vectors, called 
atoms, which are drawn from a large set called a dictionary. Unless the dictionary is orthonormal, 
the seminal approach [8] does not yield a stepwise reconstruction of the orthogonal projection of the 
signal onto a selected subspace. A variation of this approach, called Orthogonal Matching Pursuit 
(OMP) does yield the orthogonal projection [9]. Such a reconstruction is therefore optimal in the 



sense of minimizing the norm of the approximation error. However, to render a matching pursuit 
strategy suitable for discriminating signals representing different phenomena, the approach needs to 
be generalized. In order to propose the Oblique Matching Pursuit (OBLMP) method addressing this 
problem we make the following assumptions. 

• The subspace W -1- in which the signal component to be filtered lies is known. 

• The signal we wish to filter admits a unique decomposition / = fi + f%, with f\ € Vk and 
h € VV- 1 . This is equivalent to assuming / <E Vk + W 1 - with Vk n W 1 = {0}. 

• The subspace Vk can be spanned by vectors of the dictionary in hand. 

As discussed in the previous section, the reconstruction that eliminates the signal component in W 1 ' 
is f k = Ey k y V ±f. Our goal is to construct the oblique projector by using the appropriate dictionary 
vectors. We know how to update E VkW ± to E Vk+iW ± so as to account for the inclusion of an additional 
vector Vk+i- The question arises now as to how to select Vk+i giving rise to the right subspace. We 
answer this question by recourse to the consistency principle [2,6]. Considering that at iteration k the 
approximation f k of / is E Vk y V ±f, let us define the consistency error with regard to a new measurement 
w k+i as A = | (i^fe+i > / — Ev h W- L f}\- Thus to construct the approximation f k+1 = E Vk+iW ±f we 
propose to select the measurement vector wttl such that 



where J is the set of indices labeling the corresponding dictionary vectors not selected in the previous 
steps. 




(3) 



Proposition 1. If vectors w\ , % 



= 1 



. . . , k have been selected by criterion (j3J) and \ {w\ 



.k+l 
fe+1' 



f-E VkW ±f)\ + 



0, the measurement vector and the previously selected vectors w k , i 



= 1 



. . . ,k are linearly inde- 



pendent. 
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Proof. Assume that, on the contrary, |(w^J,/ — E VkW ±f)\ ^ and there exists a set of num- 
bers {ai} k =1 such that w^f\ = Yli=i a i w i- Since for the previously selected vectors the consis- 
tency condition holds, i.e. (wf,f) = (w k ,E VkW ±f), i = 1 . . . , k, we have \{w k k X\j ~ E v k W^f)\ = 
\(Ei=i a i w iJ ~ E V k W^f) \ = \T,i=i a i(( w tf) ~ { w ii E V k W^f))\ = °- This contradicts our assump- 
tion, which implies that w k ^_\ ^ J2i=i o>iW k . □ 

Proposition 2. All measurement vectors w k+1 (c.f. eq. are orthogonal to the reconstruction 

vectors selected in previous iterations. 

Proof. Every w k+1 is computed as in ^) and for i = 1, . . . , k it is true that (qe,Vi) = (u£,Vi) — 

(Pw k U£, Vi) = {u£, Ui) - (u£, Pw k Vi) = {U£, Ui) - (U£, Ui) =0. □ 

The last proposition allows us to re-state the OBLMP selection criterion (J3j) as 



w 



k +\ = argmax^K^ +1 ,/}|. (4) 



Proposition [T] ensures that, for a given tolerance 5 > 0, by stopping the selection process when 
the condition argmax^ g j | (w k+1 , f) \ < S is reached, the method only selects linearly independent 
measurement vectors. Let us assume that at iteration k + 1 the selected indices are l\, ... ,£k+i and 
denote ui i = V£. — P w ±vi i , i = 1, . . . , fc+ 1 and w k+1 , i = 1, . . . , k+ 1 to the corresponding duals. Since 
spanjn^}^ 1 = span{w k+1 } k ^ the fact that w k+1 , i = 1, . . . , k + 1 are linearly independent implies 
that U£., i = 1, . . . , k + 1 are linearly independent. Hence, as will be shown by the next proposition, 
at step k + 1 the proposed selection criterion yields a subspace Vk+i satisfying the requested property 
that V fc+ i C\W ± = {0}. 

Proposition 3. // nonzero vectors U£ t = V£. — Py V ±V£., i = 1, . . . , k + 1 are linearly independent the 
only vector in Vk+i = span{^£ which is also in W ± is the zero vector. 

Proof. Suppose that there exists g E Vk+i such that g G W ± . Hence, Py^±g = g and there exists a 
set of numbers {ftj}^ 1 to express g as a linear combination g = J2i=i biV£ r Thus X^ij 1 biP w ±V£ i = 



Yli=i ^i v iii which using the definition of U£. implies that Yli=i ^i u ti = 0- For nonzero linearly inde- 
pendent vectors this implies b{ = 0, % = 1, . . . , k + 1 and therefore g = 0. □ 

At iteration k+1 the selected indices £±, . . . ,£k+i are the labels of the atoms {v£ t }^_~i yielding the 
signal reconstruction as given by 

fe+l fc+i 
f k+1 = E Vk+lW ,f = £>* + \/K = £ C " + V (5) 

i=l i=l 

The coefficients in the last equation can be updated at each iteration according to (P) and ([2]), i.e., 

4X1 = «i>/> (6) 

cf +1 = ct-c k k X\{wtu lk+1 ), i = l,...,k. (7) 

It is appropriate to point out that these equations, as well as ([T|) and ([2]), have the identical form of 
the equations to modify the dual vectors and the coefficients in the Optimized Orthogonal Matching 
Pursuit Approach (OOMP) [10]. However, now the equations involve vectors of different nature 
yielding therefore a different approach. OOMP updating arises as the particular case, corresponding 
to Ui = V{, for which -E'v fe+ iW ± = Pv k +i- Nevertheless, since the criterion for the selection process we 
have adopted here does not necessarily minimize the norm of the residual error, OOMP is not a truly 
particular case of the new approach. On the contrary, we are introducing an alternative selection 
criterion based on the consistency principle, which could also be considered for producing yet one 
more variation of OMP. 

4 Implementation details and numerical example 

In consistence with the hypothesis itemized in Sec. [3] we consider that the subspace W -1 " is given, 
i.e. {r]i}f =1 such that W 1 - = sp&n{r]i}f =1 is known. For constructing Py V ± there are a number 
of possibilities. In the example we present here the set {rji}f =1 is linearly dependent and we have 
used the technique for dictionary redundancy elimination proposed in [14]. MATLAB code for its 
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implementation is available at [15]. The method produces a set of orthonormal vectors {4>i}iLi, 
m < n that we use to construct P w i = YllLx tyityi-i ')■ 

Given a dictionary {vi}t e j we proceed to compute vectors {ug}i e j as ui = V£ — Y2n=i tpniipmVe) ■ 
Except for the selection criterion the next steps parallel those for the implementation of OOMP but 
considering the dictionary {ui}i e j. A routine for implementation of OOMP based on Modified Gram 
Smidth orthogonalization with re-orthogonalization is also available at [15]. With very minor changes 
that routine can be used for the implementation of OBLMP. The algorithm is described below. 

Starting by assigning j£ = ue, I G J, at the first step we select the index l\ corresponding 
to the index for which (7^, f)/\ p%| | 2 is maximal and set gi = 7^/1 (7^ ||, w\ = (Zi/Ht^H and c\ = 
(w\,f). The index set J is changed to J = J At step k + 1 the sequence ji, I E J (at 

this stage J is the subset of indices not selected in the previous k steps) is orthogonalized with 
respect to q^ as: 7^ = 7^ — qk{qk->li) and, if necessary, reorthogonalized with respect to q\,...,qk 
i.e., 7£ = 7^ — Ylj=i QjiQj' id- After selecting the index Ek+l as * ne maximizer of {je, f) /W^eW 2 we set 

Qk+i = 74+1/1174+ill'^fc+l = Qk+l/lik+i and c fe+i = ( w k+iif) and compute {^ +1 }f =1 according to 
([TJ and {c^ +1 }^ =1 according to ([7]). For a given tolerance parameter 5 the algorithm is to be stopped 
when (7^, /)/||7^|| 2 < 6 for all £ £ J . The reconstructed signal is then obtained as in ([5]). 

We illustrate now the proposed method and its motivation by the following example: We assume 
the signal space to be the cardinal cubic spline space with distance 0.065 between consecutive knots, 
on the interval [0,4]. The background we wish to filter belongs to the subspace spanned by the set 
of functions r/i(x) = (x + l) -0 ' 05 * , i = 1, ...,50, a? € [0,4]. This set is highly redundant. A good 
representation of the span can be achieved by just five linearly independent functions. Actually, to 
avoid possible bad conditioning, we used only three orthonormal functions for constructing P\y± and 
verified a posteriori that this was enough for the backgrounds we were dealing with. In the first test 
the dictionary is the B-spline basis on [0,4]. We considered 100 signals, each of which was randomly 
generated as linear combination of 20 dictionary functions. One of such signals is plotted in the top 
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graph of Figure 1 added to the background. The functions which are obtained by subtracting to each 
basis function its orthogonal projection onto W 1 - are not exactly linearly dependent. However, the 
problem of constructing the duals is badly conditioned. Hence, the oblique projection onto the whole 
space does not yield the desired signal splitting. A failed attempt to separate the signal components 
is displayed by the broken line in the bottom graph of Figure 1. On the contrary, by applying the 
OBLMP approach, we could pick from the whole basis some elements spanning a subspace which 
includes the subspace in which the signal lies. Thus, as depicted in the same figure, the signal 
discrimination is successful. Equivalent results were obtained for all the others signals. In the second 
test the dictionary spanning the identical space consists of highly coherent spline atoms of twice as 
much support as the corresponding basis functions [16]. In this case out of 100 signals, randomly 
generated as linear combination of 20 dictionary functions, the OBLMP approach successfully split 90 
of them. The failures are due to the fact that, since the selection process is carried out by choosing 
a single atom at each step, in some cases it finds the right subspace by selecting a larger one which 
eventually includes the signal subspace. The construction of the duals in a larger subspace is likely 
to become faster badly conditioned when, as in the second test, the selected elements U£ are more 
coherent. On the other hand, related theoretical work [11,17-21] supports the assertion that a step- 
wise selection approach should be expected to make incorrect decisions more frequently when the 
coherence of the dictionary is larger. 

5 Conclusions 

A method, termed OBLMP, which allows for the selection of a suitable subspace for representing one 
of the signal components, and leaving aside other components of different nature, has been proposed. 
The approach evolves by stepwise selection of the subspace. The selection criterion is based on the 
consistency requirement introduced in [2] and extended in [6]. An effective implementation is achieved 
by stepwise updating of the measurement vectors yielding the appropriate oblique projector [13]. With 
regard to implementation and complexity OBLMP is equivalent to the OOMP approach [10,12]. 
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Figure 1: The top graph shows the simulated signal superposed on a background belonging to the 
subspace W 1 - = span{(x + l) _0 ' 05l }f£ 1 . The broken line of the bottom graph depicts the result of 
applying the oblique projection onto the subspace spanned by the whole B-spline basis on [0,4]. The 
continuous line in the same graph depicts the output of the proposed OBLMP. It reproduces the 
required signal. 

Since the subspace selection is performed by picking a single atom at each step, there is no guarantee 
that the required signal splitting will always be achieved. The success should depend on the nature 
of the signal components and the dictionaries spanning the subspaces for representing them. The 
given examples illustrate the fact that, as expected, the performance of the method depends on the 
coherence of the atoms resulting by subtracting from the dictionary atoms the orthogonal projection 
onto the background subspace. We hope that the results presented in this letter will stimulate further 
analysis of the proposed approach. 



9 



Acknowledgements 

Support from EPSRC (EP/D062632/1) is acknowledged. 

References 

[1] R. T. Behrens, L. L. Scharf, Signal processing applications of oblique projection operators, IEEE 
Trans. Signal Processing, 42 (1994), 1413-1424. 

[2] M. Unser, A. Aldroubi, A general sampling theory for nonideal acquisition devices, IEEE Trans. 
Signal Processing, 42, no 11 (1994), 2915-2925. 

[3] P. L. Ainsley, Observations on oblique projectors and pseudoinverses, IEEE Trans. Signal Pro- 
cessing, 45, no 7 (1997), 1886-1889. 

[4] T. Blu, M Unser, Quantitative Fourier analysis of approximation techniques. I Interpolations and 
projections, IEEE Trans. Signal Processing, 47 (1999), 2783-2795. 

[5] Xiang Yu, Lang Tong, Joint channel and symbol estimation by oblique projections, IEEE Trans. 
Signal Processing, 49, no 12 (2001), 3074-3083. 

[6] Y. C. Eldar, Sampling with arbitrary sampling and reconstruction spaces and oblique dual frame 
vectors, J. Fourier Anal. Appl, 9 no 1, (2003), 77-96. 

[7] Chun Yi Peng, Xianda Zhang, On recursive oblique projectors, IEEE Signal Processing Letters, 
12, no 6 (2005), 433-436. 

[8] S. Mallat and Z. Zhang, Matching pursuits with time- frequency dictionaries, IEEE Trans. Signal 
Processing, 41, no 12, (1993), 3397-3415. 

[9] Y.C. Pati and R. Rezaiifar and P.S. Krishnaprasad, Orthogonal matching pursuit: recursive 
function approximation with applications to wavelet decomposition, Proceedings of the 27th 
Annual Asilomar Conference in Signals, System and Computers, 1, (1993), 40-44. 

10 



[10] L. Rebollo-Neira, D. Lowe, Optimized orthogonal matching pursuit approach, IEEE Signal Proc. 
Let., 9, no. 4, (2002), 137-140. 

[11] J. A. Tropp, Greed is good: algorithmic results for sparse approximation, IEEE Trans, on Infor- 
mation Theory, 50, no 10, (2004), 2231-2242. 

[12] M. Andrle, L. Rebollo-Neira, A swapping-based refinement of orthogonal matching pursuit strate- 
gies, Signal Processing, 86, no. 3, (2006), 480-495. 

[13] L. Rebollo-Neira, Constructive updating/downdating of oblique projectors, 
http://arxiv.org/abs/math.FA/0610094t (2006). 

[14] L. Rebollo-Neira, Dictionary redundancy elimination, IEE Proceedings - Vision, Image and Signal 
Processing, 151, no 1, (2004), 31-34. 

[15] M. Andrle, L. Rebollo-Neira, "Biorthogonal techniques for optimal signal representation", 
http ://www.ncrg.aston.ac.uk/Projects/BiOrthog[ (2003). 

[16] M. Andrle, L. Rebollo-Neira, "Cardinal B-spline dictionaries on a compact interval," Appl. Corn- 
put. Harmon. Anal., 18, no 3 (2005) 336-346. 

[17] R.A. DeVore, V.N. Temlyakov, "Some remarks on greedy algorithms", Adv. Comput. Math., 5, 
no 2-3 (1996), 173-187. 

[18] R. Gribonval, M. Nielsen, "Approximation with highly redundant dictionaries", Proceedings of 
the SPIE, 5207 (2003), 216-227 

[19] A. C. Gilbert, S. Muthukrishnan, Martin Strauss, "Approximation of functions over redundant 
dictionaries using coherence", Proceedings of the fourteenth annual ACM-SIAM symposium on 
Discrete algorithms, (2003), 243 - 252. 

[20] R. Gribonval, P. Vandergheynst, "On the exponential convergence of Matching Pursuits in quasi- 
incoherent dictionaries", IEEE Trans, on Information Theory, 52, no 1 (2006), 255-261. 

11 



[21] D. L. Donoho, M. Elad, and V. Temlyakov, "Stable recovery of sparse overcomplete representa- 
tions in the presence of noise", IEEE Trans, on Information Theory, 52, (2006), 6-18. 



12 



